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ABSTRACT 



This study investigated factors associated with medical 
students performance on the United States Medical Licensing Examination 
(USMLE) Step One examination. The USMLE Step One emphasizes basic mechanisms 
and principles that are typically covered in the first 2 years of medical 
school. The study examined the relationship among student performance on Step 
One, academic achievement in medical school, and the preparation method for 
Step One. It also examined students perceptions of the efficacy of the 
medical school curriculum in preparing for Step One and compared preparation 
course participants and nonparticipants on their perceptions of preparedness 
for Step One. The study was conducted at the University of Miami School of 
Medicine, Florida, and all students who participated in a commercial 
preparation course took the Kaplan course. One hundred medical students (68% 
of the sample) responded to the survey, and of these, 32 students said they 
were preparation course participants. Student records were available for 135 
students. Of the 143 class members, 97% passed Step One on the first try. 
Academic achievement was an important factor related to Step One performance. 
The commercial test preparation course was well received by the students, but 
student performance was not enhanced by participating in the Kaplan course. 
Students thought the curriculum had prepared them well for Step One, but 
those who participated in a commercial preparation course thought it had been 
valuable. (SLD) 
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STUDENT USMLE STEP ONE PREPARATION AND PERFORMANCE 

Paper Presented at the 83rd AERA Annual Meeting in New Orleans, LA 

April 3, 2002 
By Chi Zhang, Ph.D. 

1 University of Miami School Of Medicine 

The United States Medical Licensing Examination (USMLE) is currently the sole examination 
pathway to initial licensure for allopathic physicians in the US. The three Steps of the USMLE have 
different foci and form a continuum. The USMLE Step One emphasizes basic mechanisms and principles 1 , 
which are typically covered within the basic science and pre-clinical curriculum during the first two years of 
medical school. Over three fourths of US medical schools require medical students to pass the USMLE Step 
One for promotion and/or graduation 2 . Residency program directors also view USMLE Step One scores as 
nationally standardized, objective report of information in the student profile . Many medical students take 
commercial preparation courses to help them prepare for Step One because of its importance to their careers. 
However, a 1998 survey on a random sample of Step One examinees in the US and Canada suggested that 
participation in such courses had little effect on their performance 4 . Two earlier studies concurred that 
commercial review courses had little or no impact on the medical board exam performances 5,6 . However, 
previous studies have suggested a strong correlation between student performance on Step One and 
academic achievement in medical school 4,7 . Other reported predictors for Step One performance included 
basic science pathways and MCAT scores . 



Objectives 



This investigation explored factors associated with medical students’ performance on the USMLE 



Step One. The three major objectives of this study were to: 
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(1) examine the relationships between student performance on Step One, academic achievement in the 
medical school, and the preparation method for Step One; 

(2) examine students’ perception on the efficacy of the medical school curriculum in preparing them for 
Step One; and 

(3) compare preparation course participants and non-participants on their perceptions of preparedness 
for Step One. 

Perspectives 

This study was conducted at the University of Miami School of Medicine (UMSM), a private school 
in the southeast with an enrollment of approximately 600 medical students for the 2000-01 academic year. 

Up until the 2001-02 academic year, the UMSM curriculum was a discipline-based model 7 , consisting of two 
years of basic sciences, mechanisms of disease, and clinical skills, and two years of clinical clerkships. 

All second-year medical students at the UMSM are required to pass Step One for promotion to the 
junior year. The UMSM students have approximately eight unscheduled weeks between the second and 
third years of medical school. Students must sit for the USMLE Step One prior to the beginning of the third 
year clinical clerkships. Several available commercial preparation courses for Step One are offered to 
students in May and June in the region. The preparation course of choice in 2000 was offered by Kaplan, 
which provided three weeks of full-day lecture format instruction. All of the students that participated in a 
preparation course took the Kaplan course that year. 

Methods 

A survey questionnaire was distributed to the junior medical students (N=148) in July 2000 after they 
took the USMLE Step One but before they received the Exam results. The gender makeup of the junior class 
included 71 (48%) male and 77 (52%) female. The questionnaire contained 43 multiple-choice questions 
including: (a) demographic data and the number of weeks they spent preparing for Step One; (b) factors 
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influencing their decision on preparation course participation; (c) assessment of the medical school 
curriculum; and (d) preparation methods for Step One. 

Student academic records were obtained including Step One scores, UMSM course grades, UMSM 
weighted grade point averages (GPAs) for the first two years of medical school, class GPA quartile standing, 
as well as attendance status on preparation course participation. 

The UMSM uses a numeric grading system, assigning course grades on a scale of 0-100. Weighted 
GPAs were calculated based on the individual course grades weighted by the number of credits. 

Data Collection 

One hundred medical students (68%) responded to the survey including 44 males and 56 females, 
among which 32 students claimed to be preparation course participants. Among the sample of 148 junior 
medical students of this study, 135 records of first year academic achievements (GPAs, quartiles & course 
grades) and 140 academic records for the second year were obtained from the medical school. The gaps for 
student records were due to multiple factors: dropouts, transfers and leaves of absence. Records from the 
school indicated that a total of 43 (29%) students participated in the preparation course for Step One. To 
compare with the targeted sample, the respondents had slightly higher percentages of females (56% vs. 52%) 
and preparation course participants (32% vs. 29%). 

Descriptive and inferential statistical techniques were used to analyze the data. An a level of .05 was 
used in determining statistical significance for inferential statistical analyses. 

Results 

The 2000 USMLE Step One passing score was 179. Ninety-seven percent of UMSM juniors (n=143) 
passed the 2000 Step One on their first attempt. One student who failed initially soon dropped out of 
medical school due to other academic reasons. The other four students who failed initially passed Step One 
on their later attempts. Among these five juniors, two took the preparation course and three studied on their 
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own. The UMSM mean score for the 2000 USMLE Step One was 215 ± 21, which was very similar to the 
national average (215 ± 23). 

Associations between the following factors and methods of preparation for Step One were evaluated: 
“personal learning habits”, “advice from other students” “the need for an organized schedule” “deficiency of 
the curriculum”, “reputation of the course”, “confidence in your abilities”, and “money”. “Personal learning 
habits” and “advice from other students” were the two most important factors affecting all the participants 
(100%) and nearly all the non-participants (97%) in their decision as to how to prepare for Step One. “The 
need for an organized schedule” was an important factor to all student participants but only to 59% (n=40) of 
he non-participants. “Money” was an important factor to 40% of the participants and 60% of the non- 
participants. Other factors included “deficiency of the curriculum”, “confidence in your abilities”, and 
“reputation of the course”. Overall, there was a very low correlation (Spearman p = . 152, p > .05) on the 
responses to the seven listed factors between the participants and non-participants (Table 1). 



Table 1: Factors Influencing Students’ Decision on Preparation Course Participation 



Factor 


Participants 


Non-participants 


Advice from other students 


93.8% 


88.1% 


Money 


39.7% 


59.7% 


Reputation of the course 


96.9% 


56.7% 


Personal learning habits 


100% 


97.0% 


Deficiency of the curriculum 


85.4% 


38.8% 


Confidence in your abilities 


71.0% 


83.6% 


The need for an organized schedule 


100% 


59.1% 



" important ” to each listed factor 



There was no significant difference in first-year GPAs between the prep course participants (85.79 + 
4.55) and non-participants (87.23 ± 4.35) (t = 1.75, df= 132, p = .08). Non-participants had a higher second- 
year GPA (87.78 ± 4.27) than participants (85.35 ± 4.58) ( t = 3.05, df= 138, p < .05). Chi-square test results 
suggested that there was a significant relationship between the first year class quartile standing and 
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preparation course participation (% 2 = 1 1.38, df= 3, p < .05). A significant relationship was also found 
between the second year class quartile standing and preparation course participation (x 2 = 9.846, df= 3, p < 
.05). A greater proportion of students from the lowest quartile of the first and second year GPAs took 
preparation courses. 

The Step One score correlated significantly with both the first-year GPAs (Pearson r = .111, p < .01) 
and the second-year GPA scores (Pearson r = .798, p < .05). The correlation coefficients between Step One 
score and the first-year courses in the medical school ranged from .283 (with Clinical Skills I course) to .621 
(with Biochemistry and Molecular Biology course). The correlation coefficients between Step One score 
and the second-year courses in the medical school ranged from .422 (with Introduction to Psychiatry course) 
to .694 (with Mechanisms of Disease II Module). 

Using a Mest, the difference on Step One mean scores between participants (206.00 ± 18.49) and 
non-participants (218.73 ± 21.36) was found statistically significant ( t = 3.42, df= 147, /> < .05). Non- 
participants had a significantly higher Step One mean score than participants. However, due to the 
significant difference in the second year GPA means between participants and non-participants, the pre- 
existing difference in GPAs should be adjusted for a meaningful comparison of the Step One scores between 
the two groups. By using an analysis of covariance (ANCOV A) with the second-year GPA as a covariate, 
the effect of preparation course participation was not statistically significant (F = .14 ,p — 0.71) (Table 2). 
This suggests that participation in the preparation course had no effect on student Step One performance 
after controlling for the effect of GPA. 

Table 2: ANCOVA on Difference of Step One Performance between Preparation Course Participants 
and Non-participants: 





DF 


Mean Square 


F-Value 


P- Value 


Status of Prep Course Participation 


1 


26.71 


.14 


.71 


2 nd -year GPA (Covariate) 


1 


25980.71 


131.60 


<.001 


Interaction (Participation x 2 Ild -year GPA) 


1 


43.03 


.22 


.64 


Residual 


134 


197.42 
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Overall, about 84% of the survey respondents perceived the first two years of the medical school 

curriculum to be helpful in preparing them for the USMLE Step One. Among the specific segments of the 

medical school curriculum, “pathology” (94.0%), “physiology” (89.9%), “anatomy/neuroanatomy/ 

embryology” (87.9%), and “microbiology/immunology” (84.5%) were thought to be most helpful; 

“behavioral sciences” (43.4%), “biochemistry/genetics” (46.5%), and “pharmacology” (57.6%) were thought 

to be least helpful (Table 3). There was a strong correlation between preparation course participants and 

non-participants on the perceptions of the medical school curriculum (Spearman p = .882, p < .05). 

Table 3: Student Perceptions of the Medical School Curriculum and Preparation Course in Preparing 
Them for Step One: 



Course Segment 


Medical School Curriculum 


Prep 

Course 


All 

Respondents 


Non- 

participants 


Prep Course 
Participants 


Pathology 


94.0% 


92.5% 


96.9% 


61.1% 


Physiology Portion 


89.9% 


89.5% 


90.7% 


93.8% 


Anatomy/Neuroanatomy/Embryology 


87.9% 


82.0% 


90.0% 


87.1% 


Microbiology/Immunology 


84.7% 


85.0% 


83.9% 


93.5% 


Pharmacology 


57.6% 


62.7% 


46.9% 


96.9% 


Biochemistry/Genetics 


46.5% 


41.8% 


56.3% 


90.4% 


Behavioral Sciences 


43.4% 


41.8% 


46.9% 


80.7% 



Note: The numbers shown are the percentages respondents who perceived that the medical school 
curriculum or the preparation course “ definitely ” or "somewhat” prepared them well for Step One 

Ninety-seven percent of preparation course participants perceived that the preparation course either 
“definitely” or “somewhat” helped them in preparing for the USMLE Step One. Over half (55%) of the 
participants would “definitely” recommend the preparation course to future students. About 27% of the 
participants would “somewhat” recommend the course. Only 1 8% would not recommend the course “at all”. 
Most participants found the course materials (books, CD-ROMs) to be “definitely” (53%) or “somewhat 
(41%) helpful. The live lectures were perceived to be “definitely” (69%) or “somewhat” (22%) helpful. 
Similarly, many participants believed that the course “definitely” (55%) or “somewhat (27%) prepared 
them well with “test format/test-taking strategies”. Approximately 28% of the participants felt that the 
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“amount of time in lecture” was “definitely” appropriate, and 53% of the participants thought it was 
“somewhat” appropriate. 

Non-participants perceived that the self-study method was either “definitely” (84%) or “somewhat” 
(15%) an efficient use of their time. About 59% of non-participants reported that they studied alone, while 
30.9% reported studying with another person, and 10.3% reported studying with a group. 

In terms of specific segment of the preparation course, “pharmacology” (96.9%), “physiology” 
(93.8%), “microbiology/immunology” (93.5%), “biochemistry/genetics” (90.4%), “anatomy/neuroanatomy/ 
embryology” (87.1%), and “behavioral sciences” (80.7%) were thought to be very helpful. However, only 
67.7% of the participants were in favor of the “pathology” segment (Table 3). There was a negative 
correlation between the perceived quality of the medical school curriculum and the perceived quality of 
corresponding segments of the preparation course (Spearman p = -.316). 

Most non-participants felt that they were “definitely” or “somewhat” prepared for all portions of the 
USMLE Step One - “physiology” (100%), “behavioral sciences” (100%), “pathology” (100%), 
“microbiology/immunology” (99%), “biochemistry/genetics” (97%), “anatomy/neuroanatomy/embryology” 
(96%), and “pharmacology” (94%). 

Discussions 

The results of this study indicate that academic achievement during the first two years in medical 
school is an important factor related to student performance on the USMLE Step One, which concurs with 
previous findings 4,7 . This suggests that students’ early academic performance in medical school can be used 
as a predictor for future USMLE Step One performance. Remedial studies or tutorials in the weak subject 
areas should be recommended and offered to those students with low GPAs. 

Commercial preparation courses such as Kaplan’s were well received by the participants in general. 
These courses served well for those of some students who needed an organized schedule. However, student 
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performance on Step One was not enhanced after participating in the preparation course. Kaplan’s 
preparation course appeared to offer no advantage as a remedial course to the medical students. In addition, 
this study showed that self-study can be an equally effective method to prepare for Step One for those 
students who prefer to study alone or in pairs. 

The medical students in this study felt the school curriculum prepared them well for Step One. Most 
students felt well prepared for Step One regardless of their preparation methods. Student expectations for the 
preparation course appeared to be based on their coursework experiences in the medical school curriculum. 
There was an inverse relationship between the perceived course quality of the curriculum and the perceived 
quality of corresponding segments of the preparation course. For example, pathology was the most favored 
course within the UMSM curriculum according to the preparation course participants; however, it was the 
least favored segment in the preparation course. Pharmacology was the most favored segment of the 
preparation course according to participants; yet it was one of the least favored courses with the UMSM 
curriculum by the same group. It can be postulated that preparation course participants were looking for 
added value to complement the perceived weaknesses in the medical school curriculum. 

The junior students at the University of Miami School of Medicine did well on the USMLE Step One 
in 2000. The vast majority of the junior students passed Step One on the first attempt, and the remainder 
passed it in later attempts. A few conclusions can be reached based on the findings of this study: (1) 
performance on Step One is related to academic performance in medical school and not the type of 
preparation methods; (2) students perceived value in whatever method of preparation they used; and (3) the 
strengths of the commercial preparation course were in areas of perceived curriculum weaknesses. 

Generalization of the findings from this study to other medical schools or student groups may not be 
warranted due to its scope and study samples. More investigations on this subject will bring better 
understandings of the multiple factors that affect student performance on the USMLE Step One. 
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